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ABSTRACT 

Detecting or classifying already known sparse signals con- 
taminated by Gaussian noise from compressive measure- 
ments is different from reconstructing sparse signals, as its 
objective is to minimize the error probability which describes 
performance of the detectors or classifiers. This paper is 
concerned about the performance improvement of a com- 
monly used Compressive Classifier. We prove that when the 
arbitrary sensing matrices used to get the Compressive Mea- 
surements are transformed into Equi-Norm Tight Frames, 
i.e. the matrices that are row-orthogonal, The Compressive 
Classifier achieves better performance. Although there are 
other proofs that among all Equi-Norm Tight Frames the 
Equiangular tight Frames (ETFs) bring best worst-case per- 
formance, the existence and construction of ETFs on some 
dimensions is still an open problem. As the construction of 
Equi-Norm Tight Frames from any arbitrary matrices is very 
easy and practical compared with ETF matrices, the result of 
this paper can also provide a practical method to design an 
improved sensing matrix for Compressive Classification. We 
can conclude that: Tight is Better! 

Index Terms — Compressive Classification, Sensing Ma- 
trix, Tight Frame, Equiangular Tight Frame (ETF) 

1. INTRODUCTION 

Recent years considerable progress has been made towards 
sensing and recovery of sparse signals, known as Compres- 
sive Sensing JT] Q. However, less attention has been paid 
to detecting or classifying sparse signals from compressive 
measurements, which can be stated as the problem of Com- 
pressive Detection or Compressive Classification. 

The problem of Compressive Detection and Compressive 
Classification is derived from the theory of statistical signal 
processing [3|. In this compressive scenario, Gaussian con- 
taminated sparse signals are compressed by under-determined 
linear transformations, and then the compressed measure- 
ments are used for signal detection or classification. Since 
the objective in Compressive Detection or Classification is to 
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minimize the error probability of the detectors or classifiers, 
the analysis of the influence of sensing matrices on the Com- 
pressive Detection or Classification is different from that of 
reconstructing sparse signals. 

The earliest analysis of the performance of Compressive 
Detection and Compressive Classification we ever know is 
from Davenport's publications in 2006 [4|[5 1. In these publi- 
cations the row-orthogonal random projection matrices were 
analyzed to get some performance bounds on the error prob- 
ability of Compressive Detectors and Classifiers. 

Besides, there are other publications such as Ramin Za- 
hedi et al. 's works [6 1 [7 1, in which Equiangular Tight Frames 
(ETFs, 0) were considered and proved to have best worst- 
case performance in Compressive Detection and Classifica- 
tion under the constraint that the sensing matrices are row- 
orthogonal. 

This paper is mainly concerned about the design of sens- 
ing matrices to improve the performance of Compressive 
Classification. We noticed that all these publications men- 
tioned above (|4|[5|[6|[7|) and many other related most re- 
cent researches ([9] [10]) did their analysis under the con- 
straint that the sensing matrices are row-orthogonal, i.e. the 
column vectors form an Equi-Norm Tight Frame. The reason 
why they use this constraint is for simplicity and avoiding 
the coloring of received signals between different hypothe- 
ses. While in this paper, we will show that the transfor- 
mation to Equi-Norm Tight Frames from arbitrary sensing 
matrices will make the error probability of the commonly 
used Compressive Classifier to decrease, which coincides 
with the row-orthogonal constraint commonly used above 
in (0 000). Although there are already proofs ([6|[7|) 
showing that Equiangular Tight Frames (ETFs) have best 
worst-case performance among all Tight Frames, our job is 
different from theirs. Because constructing Equi-Norm Tight 
Frames is much easier and more practical, our conclusion 
can be: Tight is Better at both improved performance and 
practical construction! 

The remainder of this paper is organized as follows: In 
Section 2, we give the problem formulation of Compressive 
Classification and the error probability; in Section 3, we 
demonstrate our main result and proof of the influence of 
Equi-Norm Tight Frames on the error probability; and in the 
last section simulation results and discussions are provided. 



2. COMPRESSIVE SIGNAL CLASSIFICATION 

Derived from the traditional theory of statistical signal de- 
tection and estimation (0), in the Compressive Classifica- 
tion scenario, the sparse signals is compressed by under- 
determined linear transformation first, then these compressed 
measurements are used to do classification. As is stated in 
Davenport's works ([4 |[5Q, the model of compressive signal 
classification can be expressed as: 
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in which Si G = {s G K , ||s||o < k} are k-sparse 
signals, i.e. the vectors with at most k nonzero elements, be- 
sides they are orthogonal to each other and have equal norms 
((« 4> «i) = OjsiUl = \\ Sj \\li ? j). y g R" is the 
received compressive measurements, n ~ A/"(0, cr 2 In) is 
the Gaussian noise contaminating the sparse signals. And 
4> G M" x w , 71 < JV is the under-determined sensing matrix, 
which has full row rank and equal column norms, and also 
satisfies RIP property ( 01110121 ). All those conditions above 
are generally used for convenience or necessity. 

According to the works of Davenport [5 1, Zahedi [6] and 
Rao (9), under the constraint of row-orthogonal (€>$ T = I) 
to the sensing matrices, the following classifier can be used: 

U = y T <Z>Si - ~\\$Si\\l 

= (y,$Si) - -{$Si,&Si), i = 1,2, • • • ,m (2) 



And the classification result is 



= argmaxjti} i = 1,2, 



(3) 



The classifier in (ffj is a most commonly used classifier, 
it is composed of a term of correlation and an minus term 
for normalization. The correlation part y T &Si also plays 
an important role in iterations of the Orthogonal Matching 
Pursuit (OMP) algorithm [13]. Since all the known analy- 
sis of Compressive Classifiers is restricted to row-orthogonal 
sensing matrices, our work is going to break this restriction. 
So the performance of the Compressive Classifier in (0 with 
more general sensing matrices that may or may not be row- 
orthogonal will be analyzed as follows. 

Theorem 1. If we use Compressive Classifier (0 and (0 to 
classify (fJJ, the error probability can be expressed as: 

^<Z>(- ~ !tiZ~ 8iW \ > l<r<m(4) 

^ 2cr||* T *(s T - Si) || 2 

Here $(x) = J*^ exp (--y) dt > and * G R nxN , n < 
N is the sensing matrix. 



Proof. The error probability is 10 

m 

V E = l-yV(U> max {t 3 }\Hi)V{H t ) (5) 

In general, the prior probability of different hypothe- 
ses can be assumed as uniformly distributed (P(Hi) = 
V(Hj),i 7^ j), and the conditional probability in (0 is 
assumed to be equal [3 | for symmetry, that is 



V{U > max {t k }\Hi) = 

l<k<.m,k^i 

V(tj > max . .{t fc }|fl» i^j 

l<k<.rn,k^j 



(6) 



then the error probability is 

V E = l-V(t T > 

= l-V(t T >U,Vi^T\H T ) 



max {ti}\H T ) 

l<i<m,i=£T 



(7) 



Thus the error probability can be analyzed with respect to the 
conditional probability V(tr > *i»Vi ^ T\Ht) under the 
Ht hypothesis. The statistics (0 under the Ht hypothesis 
have the following joint distributions 
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and 
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(* T *Si,* T *S T ) ||* T *Si||| 

Combined with the Union Bound of probability theory, the 
error probability is then 



and = P 



||*(«T-«i)||l ■ 

2<T||* r *( ST -aOlla' 
exp (— C-)di. 



1< T < m 



□ 



As a matter of fact, if there is only two hypotheses in 
the Compressive Classification problem (0, the inequality be- 
comes equality in (gj). 

So in order to decrease the error probability (0 of clas- 
sifier 01 without the constraint of row-orthogonal to sensing 
matrices and for all possible k-sparse signals s,-, we will have 
to increase 



||*(«T 



1 < T ^ i < m 



(9) 



||* T *(« T -Si)|| 2 

for all k-sparse signals st,si G A& = {s G Mr , \\s\\q < k}. 

In the cases where sensing matrices satisfying row- 
orthogonal (<&<I> T = /), (0 is then reduced to 



\ms T - Si )\\i 

* T *(s T - Si) II 



|*(»r — **)||a, l<T^i<m 

(10) 



And this is what Davenport |4|[5 1 and Zahedi [6|[7 1 analyzed 
in their publications. 



3. SENSING MATRICES AND THE ERROR 
PROBABILITY OF COMPRESSIVE SIGNAL 
CLASSIFICATION 

Although there has been plenty of works about the perfor- 
mance analysis of Compressive Detection and Classification, 
all these works have the same row-orthogonal presumption, 
and without a theoretical explanation. However, what we be- 
lieve is that there exist other important reasons for the row- 
orthogonal condition to be necessary. Here is our main result 
of this paper: 

Theorem 2. In the Compressive Classification problem (fJJ, 
If the sensing matrix $ £ R" x , n < N is transformed into 
an Equi-Norm Tight Frame, the error probability (© of the 
classifier (0 will decrease. 

Proof. According to Section 2, the error probability (0) is de- 
termined by the following expression ((9): 



*(s T - Si) 



# T *(s T — s») || 



l<T^i<m 



for all k-sparse signals sy, Sj, where <1> is an arbitrary sensing 
matrix satisfying RIP. 

According to the basic presumptions of $ in ((T), the ar- 
bitrary under-determined sensing matrix <fr G R nxJV , n < N 
has full row rank, thus the singular value decomposition of 3? 
is 

* = C/[E„ 0]V T (11) 

Here S n £ M nX " is a diagonal matrix with each element 
<fr's singular value ff, / (1 < i < n), and U £ R nx ™, 
V £ R NxN are orthogonal matrices composed of $'s left 
and right singular vectors. 

If an arbitrary equi-norm sensing matrix $ is trans- 
formed into an Equi-Norm Tight Frame <&, we do row- 
orthogonalization to its row vectors, which is equivalent as: 



3> = V~c-UY.- 1 U T $ = ^-U[I n 0]V 1 



(12) 



Where U G R" x ", V G R NxN are *'s singular vector 
matrices. In a word, row-orthogonalization is equivalent to 
transforming all singular values of $ into equal ones. Thus 
= c • I n , where c > is a certain constant for normal- 
ization. 
Then 



\\*(s T - Si )\\l 

* T *(S T - Si)|| 2 



[ I n O ]V T (s T - Si )\\ 2 (13) 



And for arbitrary sensing matrix 3? that may not be row- 
orthogonal, we have 

||*(ST-Si)||| 



|$r $(sT _ s .)|| 2 

|| [ S„ O ] V T (s T - Si )\\ 



[ Hi O ] V T{s T - Si 



(14) 



If we denote V t (st — Si) by u^ T ' l \ where u( T '' 
[ui,U2, ■ ■ ■ , un] t ■ Then < TT~4T > becomes 



[ £„ O ] V T (s T 



Si)\\ 2 2 



2 2 



[ o ]yT (sr _ Si) | 



< 



E«, 2 = 



[ I» O ] F T (S T - Si 



(15) 



The last inequality is derived from the Cauchy-Schwarz In- 
equality, combining (fT4l . (fT5T l with (flj] ). then we have 



|*(*r-«OII 



< 



|*(sr - s») || 



|* T *(s T - Si )|| 2 " ||* T *(s T -Si)|| 2 



(16) 



which means that row-orthogonalization makes (|9) larger and 
thus brings lower error probability. The condition when the 
equality holds is that 

[E= 0]V T (s T - Si ) = c-[I n 0}V T (s T - Si )(ll) 

where c > is a certain constant. 

It is obvious that the equality in ( [Tol l holds for all k-sparse 
signals St and s,, if and only if = c • I n , which means 



= c • J„ 



(18) 



So the result of ( TTST l means that when arbitrary under- 
determined sensing matrices $ G W nxN ,n < N are trans- 
formed into an Equi-Norm Tight Frame, i.e. row-ortho- 
gonalized, the equality in ( fT6l ) will hold, then the value of 
(O will get larger, which also means improvement of the 
performance of Compressive Classifier (|2). □ 

the constant c > above is an amplitude constant for nor- 
malization, with the equi-norm presumption of sensing matri- 
ces, the following corollary can be deduced: 

Corollary I. If a matrix 3? € R" xAr , n < N form an Equi- 
Norm Tight Frame, that is 4><& T = c ■ I n and \\<fii\\2 — 
\\(/>jh = -0, then c = ^tp 2 . 

Proof. If 3? has equal column norms and satisfies $4> T = 
c • I n , then 



As a result, c— —ib 2 . 



(19) 
□ 



If we let c = 1, then we can get ||<fo||2 = ||<A?||2 



yJn/N, which coincides with the results of and Q. 

Before the end of this section, some important discussions 
are believed to be necessary here. 

• Remark 1 . The result of Theorem 2 and the inequal- 
ity ( TTol l does not mean that Equi-Norm Tight Frames 



as the sensing matrices will make the error probabil- 
ity of the commonly used Compressive Classifier (f2) 
achieve an optimal lower bound. Actually the right 
side of dT6b is still dependent on $ because of the exis- 
tence of 4>'s right singular vector matrix V. Our point 
in theorem 2 is that when arbitrary sensing matrices 



pnxN 



,n < N are "tightened", or transformed 



into Equi-Norm Tight Frames, then the equality in ([16} 
holds; as a result, the error probability (3) will become 
lower, thus brings performance improvement for the 
Compressive Classifier ©. Similar discussions about 
the improvement of Equi-Norm Tight Frames to oracle 
estimators can be found in |fl4*l . which is also a good 
support of the advantage of "Tight". In a word, Tight is 
better! 

Remark 2. When the inequality ( TTol l becomes equality, 
that is when the sensing matrices satisfy 3><I> T = c • I, 
it can be proved that the error probability satisfies 



m 

V E < *(■ 



$(ST - Sj)|| 2 

2c-V2 . a 



) 1 < T < m 



which coincides with Davenport's J4] and and Za- 
hedi's |6j and J7J, where they set c = 1. So this also 
explains the benefits of using the row-orthogonal con- 
straint to do Compressive Classification. 

• Remark 3. In comparison with the the work of Zahedi 
in 1 6 1 and \7\, where Equiangular Tight Frames (ETFs) 
are proved to have the best worst-case performance 
among all Tight Frames (row-orthogonal constrained 
matrices), we just give the proof that for general under- 
determined sensing matrices, tightening can bring per- 
formance improvement for Compressive Classification. 
Our job is different from theirs, because all of Zahedi's 
analysis is based on the constraint that the sensing 
matrices are tight, or row-orthogonal, and the advan- 
tage of Equiangular Tight Frames (ETFs, [8|) is that 
ETFs have the best worst-case (maximum of the min- 
imum) performance among all Tight Frames of same 
dimensions, while our result shows that when arbitrary 
sensing matrices is "tightened", i.e. transformed into 
Equi-Norm Tight Frames, the performance of Com- 
pressive Classification will get improved. Nonetheless, 
the existence and construction of ETFs of some certain 
dimensions remains an open problem ([8]), while doing 
row-orthogonalization for any arbitrary matrix is very 
easy and practical. So our point is that Tight is Better at 
both improved performance and practical construction. 

4. SIMULATIONS 

In this section the main result of theorem 2 is verified by 
Monte-Carlo simulations. In the simulation some arbitrary 
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Fig. 1. Monte-Carlo simulation of classification error proba- 
bility using non-tight frames and tight frames 

k = 1 sparse signals are generated and classified using Non- 
Tight Frames and Tight Frames. Here we choose N = 500, 
and the error probabilities is plotted in Fig JT] corresponding 
with number of measurements n from 100 to 400 and sig- 
nal to noise ratios || 1 1 a/cr from 5 dB to 9 dB. Each error 
probability is calculated from average of 5000 independent 
experiments with 10 instances of tight or non-tight sensing 
matrices. 

The simulation shows that Tight Frames transformed from 
general Gaussian random matrices have better Compressive 
Classification performance within n's whole range, which is 
the benefits that "Tightening" brings. 

5. CONCLUSION 

This paper deals with the performance improvement of a 
commonly used Compressive Classifier (|2j. We prove that 
the transformation to Equi-Norm Tight Frames from arbitrary 
sensing matrices will make the error probability of the com- 
monly used Compressive Classifier to decrease, thus improve 
the classification performance, which coincides with the row- 
orthogonal constraint commonly used before. Although there 
are other proofs that among all Equi-Norm Tight Frames the 
Equiangular tight Frames (ETFs) achieves best worst-case 
classification performance, the existence and construction of 
ETFs of some dimensions is still an open problem. As the 
construction of Equi-Norm Tight Frames from any arbitrary 
matrices is very easy and practical, the conclusion of this pa- 
per can also provide a practical method to design an improved 
sensing matrix for Compressive Classification. In a word, we 
can conclude that Tight is Better! 
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